The International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group-4 (MPEG-4) Part 10 Advanced Video Coding (AVC) Standard/International Telecommunication Union, Telecommunication Sector (ITU-T) H.264 Recommendation (hereinafter the “MPEG-4 AVC Standard”) is the first video coding standard that employs spatial directional prediction for intra coding. Spatial directional prediction for intra coding provides a more flexible prediction framework, thus the coding efficiency is greatly improved over previous standards where intra prediction was done only in the transform domain. In accordance with the MPEG-4 AVC Standard, spatial intra prediction is performed using the surrounding available samples, which are the previously reconstructed samples available at the decoder within the same slice. For luma samples, intra prediction can be done on a 4×4 block basis (denoted as Intra—4×4), an 8×8 block basis (denoted as Intra—8×8) and on a 16×16 macroblock basis (denoted as Intra—16×16). Turning to FIG. 1, MPEG-4 AVC Standard directional intra prediction with respect to a 4×4 block basis (Intra—4×4) is indicated generally by the reference numeral 100. Prediction directions are generally indicated by the reference numeral 110, image blocks are generally indicated by the reference numeral 120, and a current block is indicated by the reference numeral 130. In addition to luma prediction, a separate chroma prediction is performed. There are a total of nine prediction modes for Intra—4×4 and Intra—8×8, four modes for Intra—16×16 and four modes for the chroma component. The encoder typically selects the prediction mode that minimizes the difference between the prediction and original block to be coded. A further intra coding mode, denoted I_PCM, allows the encoder to simply bypass the prediction and transform coding processes. It allows the encoder to precisely represent the values of the samples and place an absolute limit on the number of bits that may be contained in a coded macroblock without constraining decoded image quality.
Turning to FIG. 2, labeling of prediction samples for the Intra—4×4 mode of the MPEG-4 AVC Standard is indicated generally by the reference numeral 200. FIG. 2 shows the samples (in capital letters A-M) above and to the left of the current blocks which have been previously coded and reconstructed and are therefore available at the encoder and decoder to form the prediction.
Turning to FIGS. 3B-J, Intra—4×4 luma prediction modes of the MPEG-4 AVC Standard are indicated generally by the reference numeral 300. The samples a, b, c, p of the prediction block are calculated based on the samples A-M using the Intra—4×4 luma prediction modes 300. The arrows in FIGS. 3B-J indicate the direction of prediction for each of the Intra—4×4 modes 300. The Intra—4×4 luma prediction modes 300 include modes 0-8, with mode 0 (FIG. 3B, indicated by reference numeral 310) corresponding to a vertical prediction mode, mode 1 (FIG. 3C, indicated by reference numeral 311) corresponding to a horizontal prediction mode, mode 2 (FIG. 3D, indicated by reference numeral 312) corresponding to a DC mode, mode 3 (FIG. 3E, indicated by reference numeral 313) corresponding to a diagonal down-left mode, mode 4 (FIG. 3F, indicated by reference numeral 314) corresponding to a diagonal down-right mode, mode 5 (FIG. 3G, indicated by reference numeral 315) corresponding to a vertical-right mode, mode 6 (FIG. 3H, indicated by reference numeral 316) corresponding to a horizontal-down mode, mode 7 (FIG. 3I, indicated by reference numeral 317) corresponding to a vertical-left mode, and mode 8 (FIG. 3J, indicated by reference numeral 318) corresponding to a horizontal-up mode. FIG. 3A shows the general prediction directions 330 corresponding to each of the Intra—4×4 modes 300.
In modes 3-8, the predicted samples are formed from a weighted average of the prediction samples A-M. Intra—8×8 uses basically the same concepts as the 4×4 predictions, but with a block size 8×8 and with low-pass filtering of the predictors to improve prediction performance.
Turning to FIGS. 4A-D, four Intra—16×16 modes corresponding to the MPEG-4 AVC Standard are indicated generally by the reference numeral 400. The four Intra—16×16 modes 400 includes modes 0-3, with mode 0 (FIG. 4A, indicated by reference numeral 411) corresponding to a vertical prediction mode, mode 1 (FIG. 4B, indicated by reference numeral 412) corresponding to a horizontal prediction mode, mode 2 (FIG. 4C, indicated by reference numeral 413) corresponding to a DC prediction mode, and mode 3 (FIG. 4D, indicated by reference numeral 414) corresponding to a plane prediction mode. Each 8×8 chroma component of an intra coded macroblock is predicted from previously encoded chroma samples above and/or to the left and both chroma components use the same prediction mode. The four prediction modes are very similar to the Intra—16×16, except that the numbering of the modes is different. The modes are DC (mode 0), horizontal (mode 1), vertical (mode 2) and plane (mode 3).
The choice of intra prediction mode for each 4×4/8×8/16×16 block must be signaled to the decoder and this could potentially require a large number of bits. H.264/AVC has a particular method to signal the intra mode in order to avoid the use of a large number of bits. Taken Intra—4×4 as an example, intra modes for neighboring 4×4 blocks are often correlated. To take advantage of this correlation, predictive coding is used in H.264/AVC to signal 4×4 intra modes. Turning to FIG. 5, a current block E and its neighboring blocks A, B, and C are indicated generally by the reference numeral 500. For each current block (such as block E in FIG. 5), the encoder and decoder calculate the most probable prediction mode, which is the minimum of the prediction modes of block A and block B. If either of these neighboring blocks is not available (e.g., outside the current slice or not coded in Intra4×4 mode), the corresponding mode value for the missing block (A or B) is set to 2 (DC prediction mode).
The encoder sends a flag for each 4×4 block, pre_intra4×4_pred_mode. If the flag is equal to “1”, then the most probable prediction mode is used. If the flag is equal to “0”, then another parameter rem_intra4×4_pred_mode is sent to indicate a change of mode.
If rem_intra4×4_pred_mode is smaller than the current most probable mode, then the prediction mode is set to rem_intra4×4_pred_mode, otherwise the prediction mode is set to (rem_intra4×4_pred_mode+1).
In this way, only eight values (0 to 7) are required to transmit the chosen rem_intra4×4_pred_mode to signal the current intra mode (0 to 8).
The signaling of Intra8×8 is the same as that of Intra4×4. The prediction mode for luma coded in Intral6×16 mode or chroma coded in intra mode is signaled in the macroblock header and predictive coding of the mode is not used in these cases.
Although the most probable mode in the MPEG-4 AVC Standard is simple to compute and reduces the number of bits required for coding the prediction modes, it is based on the spatial characteristics of the neighboring blocks, not that of the current block to be coded. As a result, it is not optimal in catching the local spatial characteristics of the current block.
How to signal intra prediction mode is fundamental to the efficiency of current video encoders for intra coding. It has been observed that intra prediction may not be able to perfectly eliminate the spatial correlation of video signals, as the decoded (reconstructed) intra prediction residues tend to exhibit some spatial pattern which is highly correlated to the selected intra prediction mode. For example, for images with vertical patterns, vertical intra prediction mode tends to be selected. We have observed that the decoded residues also have vertical patterns. As previously mentioned, this is because the vertical correlation in the original signal cannot be perfectly removed by the vertical intra prediction. However, none of the existing technologies took advantage of this correlation for intra prediction mode coding.